Automatic Self-Documentation in a Mobile-Native Clinical Trial Operations System and Service Suite

ABSTRACT

A self-documenting clinical trial operations system and service suite and method, with a clinical trial operation software, a coordinator application and coordinator portal, providing information for the clinical trial operation software, a trial design services module, examining a trial design provided by a coordinator and automatically generating a software specification for a trial operations service suite, a patient application and patient portal generated from the trial design services module, for providing patient information for the clinical trial operation software, a customization option to customize at least one of the patient application and patient portal, a specification option to generate a specification document describing a design of a coordinator-customized patient application, wherein the customization and specification options are provided to the coordinator via the coordinator portal.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 16/805,683 filed on Feb. 28, 2020, which is acontinuation-in-part of U.S. patent application Ser. No. 16/778,665filed on Jan. 31, 2020, which application is a continuation of U.S.patent application Ser. No. 16/100,078 and Ser. No. 16/100,094, bothfiled on Aug. 6, 2018, and claims the benefit thereof, the contents ofwhich are incorporated herein by reference.

FIELD

This invention relates to clinical trial operations. More particularly,it relates to systems and methods for “automatically” generatingclinical trial operations software interfaces, document control, supportmenus and so forth, from initial trial protocol descriptions.

BACKGROUND

Conventional clinical trial operations systems require, at a minimum,several, months of engineering and software development effort totranslate a designed trial protocol to a working software system with afull suite of document management, respective interfaces and menus forthe trial participants, as well as for the clinicians and coordinators,etc. Because of the software manpower required, the cost and effort togo from clinical trial protocol design to clinical trial protocolimplementation is significant.

What is needed, then, is a capability to generate automaticallydocuments such as software specifications and trial protocoldescriptions, at any point in the trial design life cycle, therebyrepresenting and communicating the current state of the design for thosedocuments' respective audiences.

SUMMARY

Broadly speaking, one or more embodiments enhance a mobile-nativeclinical trial operations system/service suite so it can, via directionby a coordinator, for example, automatically generate documents thatdescribe the clinical trial framework as well as automatically generatealgorithms, interfaces and menus for use by the patient.

In one aspect of the disclosed embodiment(s), a self-documentingclinical trial operations system is provided, comprising: a clinicaltrial operation software running on at least one server; a coordinatorapplication and coordinator portal, providing information for theclinical trial operation software, a trial design services module,examining a trial design provided by a coordinator and automaticallygenerating a software specification for a trial operations servicesuite; a patient application and patient portal generated from the trialdesign services module, for providing patient information for theclinical trial operation software; a customization option to customizeat least one of the patient application and patient portal; aspecification option to generate a specification document describing adesign of a coordinator-customized patient application, wherein thecustomization and specification options are provided to the coordinatorvia the coordinator portal.

In another aspect of the disclosed embodiment(s), the above system isprovided, further comprising an entire specification option to generatean entire specification document describing a design of the entireclinical trial operations system, the entire specification option beingprovided to the coordinator via the coordinator portal, and/or whereinthe trial design services module contains an application factory systemthat automatically generates via a workflow palette and a compositionstudio's visual programming toolkit, user interface screens for aclinical trial; and/or wherein the application factory system contains averification engine, the verification engine comprising: an automaticverification framework with a test case result recorder, capturing in adocument associated inputs, state changes, outputs, and result statusfor test cases; a test plan database with at least one of coverage testswith cases targeted at executing logic paths in an object to be verifiedand document templates, containing standard formats for documents thatare generated automatically; an interactive verification framework witha document editor and supporting user interface; a test device librarycontaining testable device models; application objects and applicationcomponents to be verified; and a verification service which verifies theapplication objects and application components; and/or furthercomprising, a build engine with compilation and subsystem linkingservices; and/or wherein the build engine uses a test coverage generatoras a compiler; and/or further comprising, a composition studio providingan ability to develop applications and the application components;and/or further comprising, a document creation request control; and/orwherein the composition studio utilizes a multimedia user interfacedesign toolkit and programming design toolkit.

In yet another aspect of the disclosed embodiment(s), a self-documentingclinical trial operations service suite is provided, comprising: acoordinator application and coordinator portal, providing informationfor a clinical trial operation software; a trial design services module,examining a trial designed by a coordinator and automatically generatinga software specification for an implementation of the trial; a patientapplication and patient portal generated from the trial design servicesmodule, for providing patient information to the clinical trialoperation software; a customization option to customize at least one ofthe patient application and patient portal; a specification option togenerate a specification document describing a design of acoordinator-customized patient application, wherein the customizationand specification options are provided to the coordinator via thecoordinator portal.

In another aspect of the disclosed embodiment(s), the above suite isprovided, further comprising an entire specification option to generatean entire specification document describing a design of the entiretrial, the entire specification option being provided to the coordinatorvia the coordinator portal; and/or wherein the trial design servicesmodule contains an application factory system that automaticallygenerates via a workflow palette and a composition studio's visualprogramming tool kit, user interface screens for the trial; and/orwherein the application factory system contains a verification engine,the verification engine comprising: an automatic verification frameworkwith a test case result recorder, capturing in a document associatedinputs, state changes, outputs, and result status for test cases; a testplan database with at least one of coverage tests with cases targeted atexecuting logic paths in an object to be verified and documenttemplates, containing standard formats for documents that are generatedautomatically; an interactive verification framework with a documenteditor and supporting user interface; a test device library containingtestable device models, application objects and application componentsto be verified; and a verification service which verifies theapplication objects and application components; and/or furthercomprising, a build engine with compilation and subsystem linkingservices; and/or wherein the build engine uses a test coverage generatoras a compiler; and/or a composition studio providing an ability todevelop applications and the application components; and/or furthercomprising, a document creation request control; and/or wherein thecomposition studio utilizes a multimedia user interface design toolkitand programming design toolkit.

In another aspect of the disclosed embodiment(s), a method of generatinga specification document describing a clinical trial operations systemis provided, comprising: analyzing via an embedded mobile applicationfactory system, upon request of a coordinator through a coordinatorapplication and a coordinator portal of a trial operations servicesuite, a design of applications and services of the clinical trialoperations system; and recording results of the analysis in aspecification document using a template document.

In yet another aspect of the disclosed embodiment(s), the above methodis provided, wherein the design of applications and services of theclinical trial operations system is for a patient application; and/orfurther comprising: iterating over workflows in the design of thepatient application to generate traversal paths for each workflow,wherein each traversal path is a function of user input values andsimulated to determine an associated sequence of screens forpresentation to a user when a traversal path is executed, capturing inthe specification document a graph of corresponding flow annotated witheach screen presented and the associated user input values; andfiltering duplicate results to minimize a length of the specificationdocument; and/or wherein the analyzing and recording steps furthercomprise: iterating over workflows in a design of all applications, allservices, gateways, and analytic modules of the trial operations servicesuite, to generate traversal paths for each workflow, wherein eachtraversal path is a function of input values from users, other systemelements, and connected external elements, each traversal path beingsimulated to determine an associated, sequence of screens to a user,signals to other system elements, and signals to connected externalelements when a traversal path is executed; capturing in thespecification document a graph of corresponding flow annotated with eachscreen presented and signal sent along with associated input values; andfiltering duplicate results to minimize a length of the specificationdocument.

The preceding Summary is intended to serve as a brief introduction tovarious features of some exemplary embodiments. Other embodiments may beimplemented in other specific forms without departing from the scope ofthe disclosure

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a schematic block diagram of an exemplary embodimentof a clinical trial operations system;

FIG. 2 illustrates a schematic block diagram of an exemplary embodimentof a trial operations service suite in a clinical trial operationssystem;

FIG. 3 illustrates an exemplary workflow and corresponding userinterface screens which may be included in a web or mobile deviceapplication in a clinical trial operations system;

FIG. 4 illustrates a schematic block diagram of an exemplary applicationfactory system disclosed in U.S. Pat. No. 8,719,776, enhanced asdisclosed in U.S. patent application Ser. No. 16/100,078, encapsulatedwithin a trial design services element of a trial operations servicesuite in a clinical trial operations system;

FIG. 5 illustrates a process distributed among a coordinatorapplication, coordinator portal, and other elements of a trial operatorsservice suite in a clinical trial operations system according to anexemplary automatic documentation embodiment;

FIG. 6 illustrates an exemplary method whereby a trial operationsservice suite in a clinical trial operations system may automaticallyexamine the trial design and generate a trial protocol description;

FIG. 7 illustrates an exemplary method whereby a trial operationsservice suite in a clinical trial operations system may automaticallyexamine the trial design and generate a software specification for theservice suite itself and the applications that interact with; and

FIG. 8 and FIG. 9 illustrate an exemplary workflow, and its traversalpaths as may be generated by the methods illustrated in FIG. 6 and FIG.7 .

DETAILED DESCRIPTION

The following detailed description describes currently contemplatedmodes of carrying out exemplary embodiments. The description is not tobe taken in a limiting sense but is made merely for the purpose ofillustrating the general principles of some embodiments, as the scope ofthe disclosure is best defined by the appended claims.

Various features are described below that can ac be used independentlyof one another or in combination with other features.

Introduction

The mobile-native clinical trial operations system and service suite(s)disclosed in U.S. patent application Ser. No. 16/100,078 and Ser. No.16/100,0941 (referred to hereinafter as “the base patentapplication(s)”) provides a number of capabilities: such that thepatients and their doctors who participate in a drug or device clinicaltrial benefit from automatic secure collection and storage of relevantdata, such that each trial's investigators may securely receiveappropriately anonymized (also referred to as de-identified) versions ofthat data, potentially in real time if allowed by the trial protocol,with automatic secure logging of data access, and may securely recordcorresponding research activities, observations, and conclusions in afully traceable lab notebook tied to the trial, its data, and the dataaccess logs; such that each trial's operational coordinators may designtrial enrollment (including qualification and consent), engagement, datacollection, and other information flow protocols quickly and easilywithin hours, and automatically create and deploy corresponding web andmobile device applications within minutes, without baying to hire anengineering team to spend weeks or months translating the design intosoftware; such that each trial's risk and compliance coordinators mayanalyze and extract traceability and accountability records as required;and such that all participants are able, if and as permitted by thetrial protocol, to communicate with one another securely.

Among the traceability and accountability records not explicitly namedin the cited disclosure are detailed descriptions of the trial protocol,that is, a complete set of the screens presented, data collected,decision paths supported, and interactions allowed within a particularclinical trial, with respect to all participants including patients,clinicians, and investigators. Such documents are typically composedduring trial design by experts in the subject of the trial as well asthe trial design process itself. Two significant uses of the trialdesign documentation prior to trial commencement are, first, to obtaininstitutional review board (IRB) approval for the trial and, second, tospecify the tools such as software programs to be developed forsupporting the trial. In this “waterfall”-style process, the trialprotocol can be created and perhaps iteratively refined by trialprotocol experts in advance of IRB approval and tool developmentactivity, allowing a confident handoff to the tool development experts.

In view of this, the disclosed mobile-native clinical trial operationssystem and service suite provides in particular the capability for trialprotocol designers to express the trial design directly and naturallyusing a design studio that can then automatically generate the necessarytools—that is, software in the form of service suite and userapplications. In this “agile”-style process, the need for aspecification handing off the trial design to tool developers may bebypassed, but the need for a trial protocol description supporting FRBapproval remains. Further, even though a software specification documentmay not be needed for tool development, one may be needed for review ofthe as-built system service suite and applications—by softwareengineering professionals in conjunction with IRB approval or regulatoryprocesses.

I. Hardware Architecture

FIG. 1 illustrates a schematic block diagram of a clinical trialoperations system 100 according to an exemplary embodiment. FIG. 1 is acopy of FIG from the base patent application(s), and its descriptionhere is also copied therefrom.

Data network 101 may interconnect the various elements of system 100.The network 101 may include one or more private networks as well as oneor more public networks including the Internet. Constituent networks mayinclude cellular mobile radio networks using various standards such asLTE; Wi-Fi networks serving public facilities such as airports, shoppingcenters, hospitals; Wi-Fi networks serving private facilities such ashomes, labs, offices, and clinics; wired networks such as Entremetswithin private facilities such as homes, labs, offices, and clinicscable or DSL access networks; and any other form of data network.

Each category of participant may be considered to be a user ofcorresponding elements in system 100. A patient who participates in aclinical trial supported by system 100 may use a patient personalcomputing device 104, which may be a mobile device such as a tablet orsmartphone, or a traditional computer such as a laptop or desktop, andwhich may connect to data, network 101 via an interface 114 that may beimplemented using any of numerous technologies as listed above orotherwise.

Patient application 140 may include software executed by the patientpersonal computing device 104 that may provide user interface (UI) andoperational functions supporting the patient's interaction with system100. Patient workstation 104 may also execute other softwareapplications that fall outside system 100, in addition to, or in placeof, patient application 140. For certain types of study, it may benecessary to incorporate health and environment information from sensorssuch as blood pressure monitors, heart rate monitors, personal fitnessbands, smart watches, smart home products, and/or other similar devices.

For certain kinds of study, it may also be appropriate to incorporatehealth, event, and environment information from study-specific sensorsprovided to the patient, such as a custom drug dispenser that detectsthe time and conditions when a patient consumes a dose. Such devices arerepresented in the diagram by patient sensor(s) 145. In addition tostudy-specific sensors, such patient sensors may include biometricsensors (e.g., heart rate monitors, blood pressure sensors, blood sugarsensors, thermometers, etc.), environmental sensors (e.g., humiditysensors, temperature sensors, etc.), and/or event other event sensors.Furthermore, such sensors may include data, received from a patient(e.g., indication of pain level or other appropriate factor, loggingmedication times and/or amounts, etc.).

Patient sensor(s) 145 may communicate with the patient personalcomputing device 104 via an interface 141 in order to provide sensorreadings and related information. For each device represented by patientsensor(s) 145, interface 141 may include a wired connection such as aUSB cable, and/or a wireless connection such as Bluetooth or Wi-Fi.

A doctor, advanced practitioner, or nurse who treats or overseestreatment of a patient participating in a clinical trial supported bysystem 100 may use a clinician workstation 105, which may be a mobiledevice such as a tablet or smartphone, or a traditional computer such asa laptop or desktop, and which may connect to data network 101 via aninterface 115 that may he implemented using any of numerous technologiesas listed above or otherwise.

Clinician application 150 may include software executed by clinicianworkstation 105 that provides UI and operational functions supportingthe clinician's interaction with system 100. Clinician workstation 105may also execute other software applications that fall outside system100, in addition to, or in place of, clinician application 150.

For certain kinds of study, it may be necessary to incorporate healthand environment information from sensors such as blood pressuremonitors, heart rate monitors, thermometers, CT scanners, MRI scanners,and/or other diagnostic devices. Such devices are represented in thediagram by clinical sensor(s) 155. Clinical sensor(s) 155 maycommunicate with clinician workstation 105 via an interface 151 in orderto provide sensor readings and related information. For each devicerepresented by clinical sensor(s) 155, interface 151 may include a wiredconnection such as a USB cable or Ethernet network, a wirelessconnection such as Bluetooth or a Wi-Fi network, or a combination ofsuch connections.

A technician, scientist, or other researcher who analyzes data and drawsconclusions regarding the results of a clinical trial supported bysystem 100 may use an investigator workstation 106, which may be amobile device such as a tablet or smartphone, or a traditional computersuch as a laptop or desktop, and which may connect to data network 101via an interface 116 that may be implemented using any of numeroustechnologies as listed above or otherwise.

Investigator application 160 may include software executed byinvestigator workstation 106 that provides UI and operational functionssupporting the investigator's interaction with system 100. Investigatorworkstation 106 may also execute other software applications that falloutside system 100, in addition to, or in place of, investigatorapplication 160.

A supervisor, designer, trial manager, site manager, risk manager,compliance manager, or other administrator who oversees one or moreaspects of a clinical trial supported by system 100 may use acoordinator workstation 107, which may be a mobile device such as atablet or smartphone, or a traditional computer such as a laptop ordesktop, and which may connect to data network 101 via an interface 117that may be implemented using any of numerous technologies as listedabove or otherwise.

Coordinator application 170 may include software executed by coordinatorworkstation 107 that provides user interface and operational functionssupporting the coordinator's interaction with system 100. Coordinatorworkstation 107 may also execute other software applications that failoutside system 100, in addition to or in place of coordinatorapplication 170.

The system 100 may utilize a trial operations service that providescapabilities with which each participant may interact via the respectiveapplication. A distinct trial operations service may be used for eachseparate clinical trial in order to ensure strong data privacyprotection, even when a single organization is operating multiple trialsand therefore multiple trial operations services Such an approach alsoprevents propagating any faults that may occur in one instance to otherinstances, thereby ensuring that no single fault compromises multipletrials.

The example of system 100 includes two kinds of trial operationsservice, but for any particular trial only one of those types is to bedeployed, depending on the preference of the organization operating thetrial.

A cloud-based trial operations service 120, which may include softwarefunctions executing in a cloud computing platform 102, may be selectedby an organization that prefers to outsource its information technologyoperations to an expert in that field. For example, a biopharmaceuticalresearch lab may prefer not to distract itself from its core competency,or a government research lab may require implementation using theservices of a Federal Risk and Authorization Management Program(FedRamp)-approved provider.

An enterprise trial operations service 130, which may include softwarefunctions executing in an appliance computing platform 103, may beselected by an organization such as a large company with multiplecapabilities that prefers to keep information technology operations inits own facility and network. For example, a medical device manufacturerma already have information technology competency due to the electronicand software-driven nature of its devices. Other reasons may also existfor choosing one or the other deployment option for the trial operationsservice.

One of ordinary skill in the art will recognize that system 100 may beimplemented in various different ways without departing from the scopeof the disclosure. For instance, the various elements of the system maybe arranged in various different ways. As another example, variousdevices or elements may be implemented using multiple devices orsub-elements. Likewise, in some embodiments multiple devices or elementsmay be combined into a single device or element. In addition, variousother elements may be included and/or various listed elements may beomitted in some embodiments.

II. Services

Many clinical trials take place in multiple locations, with separatepersonnel who may, or may not interact with one another. In each of theparticipant categories described above, individual participants may beassociated with one or more such locations, or sites, such that dataaccess and communication permissions may be constrained to remain withineach site For example, a coordinator in the trial manager role may havepermission to access data for all sites, while a coordinator in a sitemanager role may only have permission to access data for the assignedsite. However, in clinical trial operations system 100 all sitesassociated with a particular clinical trial are supported by the sametrial operations support service 120 or 130, thus ensuring commonalityof trial data and procedures across all sites.

Enterprise trial operations service 130 and cloud-based trial operationsservice 120 may be similar, and both appliance computing platform 103and cloud computing platform 102 provide similar capabilities. A unifiedview detailing these is provided in FIG. 2 , which illustrates aschematic block diagram for a trial operations service suite 200 andcomputing platform 210 in a clinical trial operations system accordingto an exemplary embodiment. FIG. 2 is a composite of FIG. 2 and FIG. 3from the base patent application(s), and much of its description here isrepeated therefrom.

The mapping of trial operations service suite 200 modules to both cloudtrial operations service 120 and enterprise trial operations service 130is represented by the long upper bracket. The mapping of computingplatform 210 modules to both cloud computing platform 102 and enterprisecomputing platform 103 is represented by the short lower bracket.Computing platform 210 is not further detailed here; it may besubstantially as detailed in the base patent application(s).

Because clinical trial operations are subject to strict regulatoryrequirements regarding information security and privacy, computingplatform 210 must be carefully selected and configured with featuresthat accommodate those requirements. Encryption of data at rest and inmotion, role-based data access permission capabilities, and bullet-proofcommunication stacks are all important. A cloud computing platform 102may be used only if it explicitly supports HIPAA compliance, which inaddition to the features listed above also requires physical securityand strong partitioning among customers so unrelated applications cannotinteract with one another either intentionally or unintentionally. Anorganization choosing to deploy trial operations service suite 200 in anappliance computing platform 103 is obliged to satisfy these physicalsecurity and related requirements itself.

Security configuration and management authentication database 201 mayprovide a data repository specifically for information about theparticular trial operations service suite 200 and its deployment incomputing platform 210. Configuration data may include, but is notlimited to, such persistent knowledge as how it relates to the networkin which it resides, how the various function modules are deployedacross potentially multiple physical and/or virtual computing machines,and licensed capacity. Security configuration data in particular mayprovide the Mathematical basis of trust for establishing communicationpaths at the level of attaching the services hosted in trial operationsservice suite 200 to other services elsewhere in the network. Managementauthentication data may provide the credentials and permissionsnecessary to protect access to management functions.

Operational databases 202 may provide multiple data repositoriesspecific to the clinical trial supported by trial operations servicesuite 200. This group of databases is architecturally distinct fromsecurity configuration and management authentication databases 201 forreasons of information privacy, data capacity differences, andtransaction capacity differences. A detailed description of the specificdata modules within operational databases 202 may be found in the basepatent application(s).

Beyond that foundation, the functions of trial operations service suite200 may fall into a few major groups The first of these includes themanagement functions, which may provide ways to configure and controlthe trial operations service suite 200 and the underlying computingplatform 210. Management server 203 may form the basis of the managementfunction group, providing a network interface and an operatingenvironment for the various management services. Management server 203may include such components as a web server, a secure shell interface, afile transfer protocol server, and related capabilities typically usedin the management of information technology systems. Management services230 are not further detailed here; this element may be substantially asdetailed in the base patent application(s).

The next major group of functions are the internal relay services 281,282, and 283 and the external gateways 287, 288, and 289. Relay servicesprovide capabilities for interaction among services of trial operationsservice, suite 200. Gateways provide capabilities for communication anddata access between trial operations service suite 200 and affiliatedexternal systems, or among multiple related trial operations servicesuites 200. These functions may be substantially as detailed in the basepatent application(s).

The next major group of functions is the data analytics framework 290and its modules 2901, 292, and 293. These may provide an environment forautomatic processes that analyze different types or multiple types ofdata as it comes into the system, looking for correlations indicatingsome important actionable fact. These functions may be substantially asdetailed in the base patent application(s).

The final major group of functions in trial operations service suite 200may provide access for various types of user in the trial communityserved by trial operations service suite 200. Each user category may beprovided with a respective portal 204, 205, 206, or 207, which mayinclude a web server dedicated to that category of users and which maysupport highly secure information presentation and interaction formultiple users within that category.

Multiple user roles may also be supported within each category, as wellas site assignments for each user, depending on data access permissionsencoded in a provisioning and authorization data subset of operationaldatabases 202. For each user category, support for a variety ofcapabilities may be encapsulated within a corresponding set of services240, 250, 260, or 270 built atop the corresponding portal 204, 205, 206,or 207, partitioned from the others to prevent inappropriate informationcrossover, and interacting with the corresponding application 140, 150,160, or 370 of system 100. Each user category's portal and services aredescribed below.

Patient portal 204 and patient portal services 240 may provide accessfor patients participating in the supported clinical trial, interactingwith patient application 140. The patient user category may include useraccounts not only for a particular patient participating in a clinicaltrial, but also any of that patient's family members or representativessuch as a formally declared HIPAA agent or a holder of healthcare powerof attorney who the patient has designated to have access to the system.Each patient account may have specific permissions to view data, updatedata, create diary and journal entries, receive notifications, and thelike, depending on the corresponding records in a provisioning andauthorization data subset of operational databases 202 as established bythe patient, or the trial protocol as established by a coordinator.Patient portal 204 may provide, through patient application 140, a baseview whereby the patient user can see account status, see overviews ofeach available service such as whether unviewed notifications exist, andselect an available service to activate.

Patient portal services 240 may include any or all of the services showndepending on the requirements of the particular clinical trial beingsupported by trial operations service suite 200. Additional services notshown or described may also be created within this service framework.

Patient portal services 240 may include a group of enrollment services241, a notification service 242, a group of secure communicationservices 243, a group of engagement services 244, and a group of datacapture services 245. Each of these services may be substantially asdetailed in the base patent application(s).

Clinician portal 205 and clinician portal services 250 may provideaccess for clinicians with one or more patients participating in thesupported clinical trial, interacting with clinician application 150.The clinician user category may include user accounts not only for adoctor treating a particular patient participating in a clinical trial,but also any of that doctor's staff partners, associated advancedpractitioners, or nurses who also participate in treatment or care of -aparticipating patient. Each clinician account may have specificpermissions to view data, update data, create diary entries, receivenotifications, and the like, depending on the corresponding records inprovisioning and authorization data subset of operational databases 202as established by the primary clinician, or the trial protocol asestablished by a coordinator.

Clinician portal 205 may provide, through clinician application 150, abase view whereby the clinician user can see account status, seeoverviews of each available clinician-specific service such as whetherunviewed notifications exist and select an available service toactivate, view a list of associated patients participating in thesupported clinical trial and corresponding overviews of patient-relatedservices, and select a patient and service from that list.

Clinician portal services 250 may include any or all of the servicesshown depending on the requirements of the particular clinical trialbeing supported by trial operations service suite 200; additionalservices not shown or described may also be created within this serviceframework.

Clinician portal services 250 may include a group of enrollment services251, a group of notification services 252, a group of securecommunication services 253, a group of engagement services 254, and agroup of data capture services 255. Each of these services may besubstantially as detailed in the base patent application(s).

Investigator portal 206 and investigator portal services 260 may provideaccess for investigators handling the research and analysisresponsibilities in the supported clinical trial, interacting withinvestigator application 160, and may be substantially as detailed inthe base patent application(s).

Coordinator portal 207 and coordinator portal services 270 may provideaccess for people in charge of managing various aspects of the trial,interacting with coordinator application 170. The coordinator usercategory may include user accounts for various individuals with distinctor overlapping responsibilities, including but not limited to trialdesign, everyday trial operations, and compliance Each coordinatoraccount may have specific permissions to view data, update data, changethe trial design, send notifications, and the like, depending on thecorresponding records in a provisioning and authorization data subset ofoperational databases 202 as established by the primary coordinator.Coordinator portal 207 may provide, through coordinator application 170,a base view whereby the coordinator user can see account status, seeoverviews of each available coordinator-specific service, and select anavailable service to activate.

Coordinator portal services 270 may include any or all of the servicesshown depending on the requirements of the particular clinical trialbeing supported by trial operations service suite 200; additionalservices not shown or described may also be created within this serviceframework.

Coordinator portal services 270 may include a group of enrollmentservices 272, a group of secure communication services 273, a group ofnotification services 274, and a group of data analysis services 275,all of which may be substantially as detailed in the base patentapplication(s).

Coordinator portal services 270 may also include a′group of trial designservices 271, which may support configuration of trial operationsservice suite 200 for the particular clinical trial being supportedTrial design services 271 may be used both at the beginning of a trialto set its initial design, and during the trial to change aspects of itsdesign that may need adjustment as the trial progresses. Trial designservices 271 may provide selections to configure whether specificservices are available in each group of portal services 240, 250, and260, such as which communication modes may be activated, as well asother configuration items such as whether gateways 287, 288, and 289 areactive and if so with what other systems they may interoperate, whatkind of inter-user communication, notification, and invitation pairingsare permitted, and what kind of data analytics are activated.

Trial design services 271 may also provide capabilities for designingdetailed user interaction procedures such as screen layouts, order ofpresentation and text for participant qualification and informed consentprotocols, supported sensors both mandatory and optional, sensor datacollection schedules, branding imagery and wording, wording of internaland external notification templates, wording of invitation templates,structure of surveys and status queries, structure and emphasis of diaryor journal entry templates, structure and emphasis of lab notebook entrytemplates, supported data access query templates, data access resultdisplay formats, and every other aspect of user interaction supported byapplications 140, 150, and 100. Trial design services 271 may, evenalter the look and capabilities of coordinator application 170 itself.

Trial design services 271 may also provide capabilities forautomatically generating external documentation of a particular trialdesign, in accordance with the present disclosure and described in thecontext of FIG. 5 , FIG. 8 , and FIG. 7 .

FIG. 3 illustrates an exemplary workflow and corresponding exemplaryuser interface screens which may be included in a patient application140. Example workflow 380 may be created using the application factorysystem embedded in trial design services 271, and customized to aspecific clinical trial according to the dictates of its protocol usingthe Flowable-based workflow palette in conjunction with the compositionstudio's visual programming toolkit. Similarly, example user interfacescreens 390 may be created using the application factory system embeddedin trial design services 271, and customized to a specific clinicaltrial according to the dictates of its protocol using the variouspalettes available in the composition studio's design toolkit. Note thatthe exact nature of the workflow, shown in FIG. 3 is a simple patientquestionnaire, which is visibly related by the nature of its depictedimagery and text to the clinical trial field of use pertinent in thepresent disclosure. However, it is described here in terms of itslogical and structural symbology in order to focus on the capability ofbuilding relevant workflows rather than on any specific workflow topic.A coordinator user of trial design services 271 may create any number ofsuch workflows and user interfaces to represent the trial protocol,which may then in turn be assembled into patient, clinician,investigator, or coordinator applications 140, 150, 160, or 170respectively.

The representation of example workflow 380 and its example userinterface 390 thus includes a number of symbols with specific meaningsin the context of trial design services 271. Workflow label 381 and userinterface label 391 distinguish between the logical and graphicalspecifications, while workflow name 387 and user interface name 397contain text strings created by the designing coordinator user todistinguish this workflow and its user interface from others. The samevalue, “Ask About Symptoms” in the example, is shown for both names 387and 397 to indicate that this workflow 380 and this user interface 390are related to one another.

Of particular relevance to the present disclosure are workflow attribute388 and user interface attribute 398, which both indicate that theelements of this workflow 380 and this user interface 390 are availableoffline. That is, their structure and content may be pushed toapplication 140 or 150 for use even when there is no connection to thecorresponding portal services 240 or 250. Refer to FIG. 4 and later foradditional details on this capability.

The flow aspect of example workflow 380 begins at start symbol 331 andis complete at end symbol 335. Other symbols not shown may representwaiting for a message from an external entity or suspending for a periodof time. To get from start to end, a variety of actions, decisions, andtransitions may occur. Each action block 382 provides an opportunity forthe application implementing example workflow 380 to do somethinguseful, which typically may be summarized using the action block name321. The actual details of what the action block 382 does may bespecified by invoking a composition studio canvas and palette thatcorresponds to the action block type. The action block name 321 may alsomatch a named element in the corresponding canvas, so as to specify adefinitive relationship between the workflow action block and theunderlying action. Each transition 383 may provide a time-ordered flowfrom a start symbol 331 or action block 382 to another action block 382,an end symbol 335, or a decision block 332. Each decision block 132, ofwhich only one is present in FIG. 3 , may provide a branch in the flowbased on a data item identified by the preceding action block 382. Eachpossible value of the data item may then be used as a transition name334 associated with a corresponding named transition 333, such that thedecision block 332 directs the flow along the corresponding pathaccording to the selected value. Note that the only type of decisionblock 332 shown in FIG. 3 is similar in meaning to a CASE statement in atraditional computer programming language. Other decision block typesmay be used, each one represented using a different symbol not shown,including such common operations as logical and numerical comparisons,as well as a multipath operation that splits the flow into two or moresimultaneous sequences.

Three types of action block are depicted here using distinct icons torepresent each, and others may be inferred from the composition studiodescriptions in the base patent application(s) and in U.S. Pat. No.8,719,776, which together describe more than this number ofcanvas/palette combinations. Action block type 322 may manipulate data,such as reading a variable in the type 322 action block 382 labeled“Pick Question Style” or writing a variable in the type 322 action block382 labeled “Record Response.” Within the composition studio of trialdesign services 271, selecting or creating an action block 382 of type322 may invoke the data palette of the data and logic canvas in order tospecify the pertinent data items and what is to be done with them. Itshould be noted that multiple “types” of data manipulation can beperformed by the composition studio, the types and action coming fromlinked or incorporated toolkits, non-limiting examples being multimediauser interface design and programming; design toolkits, additionalexamples being found in the base patent application(s). This palette andcanvas are not further described here; refer to the aforementionedantecedents for details.

Action block type 323 may cause the presentation of a correspondingscreen or subscreen according to the relationship indicated via theaction block name 321. Within the composition studio of trial designservices 271, selecting or creating an action block 382 of type 323 mayinvoke the user interface canvas to specify the corresponding userinterface elements. More detail regarding the construction ofcorresponding screens follows in the context of example user interface390.

Action block type 324 may cause the execution of an algorithm,effectively a subroutine that matches the action block name 321. Withinthe composition studio of trial design services 271, selecting orcreating an action block 382 of type 324 may invoke the logic palette ofthe data and logic canvas in order to specify the algorithm to beexecuted. Any operation that may be expressed in the data and logiccanvas may be incorporated in the corresponding algorithm; thus anaction block 382 of type 324 may perform calculations, interact withother entities through external communication, start or stop sensors oractuators in the mobile device running application 140 or 150, or anyother operation available in that portion of the composition studio.This palette and canvas are not further described here; theaforementioned antecedents provide details.

As previously noted, example user interface 390 is associated withexample workflow 380 through the common value in their respective names387 and 397. In the user interface canvas, a coordinator user mayarrange passive graphical elements such as text 343 and icons 344, andactive graphical elements such as selectors 345 and buttons 346. Thesearrangements may be grouped into fixed areas such as header 341 andfooter 342 so that they appear in every variation of screen 394, or theymay be grouped into subscreens 353 linked to a variable area 395 bysubscreen relationships 351 so that the appropriate visual and controlarrangement may be selected according to workflow logic. Subscreen names352 would then be used to associate each subscreen 353 with an actionblock of type 323 by matching its action block name 321.

One skilled in the art will appreciate that the fixed areas header 341and footer 342 in screen 394 may have as easily been designed usingadditional variable areas 395 and accompanied by additional subscreens353 with corresponding name relationships 352 to additional actionblocks 382 of type 323. Further, the specific decisions regardingstructure and design of a workflow and its user interface are entirelyat the coordinator user's discretion and may take any conceivable form,including more or fewer elements in different orders with simpler ormore complex logic. Similarly, the values of names 387, 397, 321, 334,and 352, as well as the content and form of graphical elements such astext fields 343, images 344, and buttons 345 and 346, are also designchoices that may be exercised by the coordinator user. One skilled inthe art will also appreciate that the range of logic and graphicelements is not limited to those described here. The full set ofcapabilities supported by the application factory system embedded intrial design services 271 may be utilized in designing real workflowsand user interfaces for a specific clinical trial. Further, it should benoted that the specific graphic style of each symbol depicted in FIG. 3is important only to the extent that it evokes the workflow or userinterface element it is meant to represent. Other symbols and symbolpositions may be used equally as well within the scope of the presentdisclosure.

In an exemplary embodiment, trial design services 271 may be implementedby embedding in trial operations service suite 200 an enhanced versionof the application factory system disclosed in U.S. Pat. No. 8,719,776,which is incorporated herein by reference. Enhancements in this regardmay be substantially as detailed in U.S. patent application Ser. No.16/100,078 and Ser. No. 16/100,094 (the base patent applications), aswell as the further enhancements provided in the present disclosureregarding, automatic documentation generation described in the contextof FIG. 4 , FIG. 5 FIG. 6 and FIG. 7 . Additional detail regarding thisembedment as well as detail regarding the automatic document generationfeature are shown in FIG. 4 .

Each of the application factory system elements, numbered 1xx in FIG. 1of US. Pat. No. 8,719,776, is depicted here in FIG. 4 numbered 271xx,highlighting the embedment in trial design services 271. Certainelements, particularly the major subsystems composition studio 27120,distribution center 27130, and build engine 27140, as well as theinterfaces among them 27124, 27142, 27132, and 27143 may besubstantially as described in U.S. Pat. No. 8,719,776, where they arenumbered 120, 130, 140, 124, 142, 132, and 143 respectively, as enhancedby the corresponding descriptions in the base patent application(s).Similarly, the various applications 27101, application components 27102,generic mobile device model 27103, and specific mobile device models27104 may, be substantially as described in U.S. Pat. No. 8,719,776,where they are numbered 101, 102, 103, and 104 respectively, except thatin the present disclosure applications 27101 and application components27102 may be specifically pertinent to the field of clinical trialsupport, where their archetypes 101 and 102 may be pertinent to anyfield.

Verification engine 27150 is enhanced in the present disclosure,compared to its archetype verification engine 150 in U.S. Pat. No.8,719,776. While structurally quite similar, each element ofverification engine 27150 includes additional functional and datahandling capabilities with respect to the new document generation andediting feature Pertinent functional elements are shown in the figure,while data handling enhancements are pervasive and not specificallydepicted. The latter may generally occur as additional fields in datastructures used inside each function as well as in communication amongthe functions and on interfaces 27125, 27152, 27154, and 27153 betweenverification engine 27150 and its peer subsystems.

Within verification engine 27150, the functional elements networkcommunication support module 27501, computation resource managementmodule 27502, objects to be verified 27510, test device library 27540,and verification service 27560 may generally be substantially asdescribed in U.S. Pat. No. 8,719,776 with respect to its FIG. 5, wheretheir archetypes are numbered 501, 502, 510, 540, and 560 respectively,although the data handling enhancements described above are incorporatedin each of those elements. Similarly, functional elements from FIG. 5 ofU.S. Pat. No. 8,719,776 not shown in automatic verification framework27520, interactive verification framework 27530, and test plan database27550 are omitted from FIG. 4 for clarity but may be inherited intactfrom FIG. 5 of U.S. Pat. No. 8,719,776, where they are numbered 520,530, and 550 respectively. New data handling elements are added for eachof those modules as well, as necessary to effect the features of thepresent embodiments

Several new modules are introduced in verification engine 27150. Withininteractive verification framework 27530, document editor 27533 mayprovide a coordinator user the ability to annotate or change anautomatically generated document. This module may incorporate commontext editing tools as well as specializations that facilitate placementand highlighting of captured test results such as screenshots, whileremoving any ability to alter them materially. Coupled with priorcapabilities of this framework to execute and record specific testcases, document editor 27533 may be used to ensure completeness orimprove readability of documents generated automatically.

Within automatic verification framework 27520, test case result recorder27524 augments prior capabilities to drive execution of test cases andanalyze their success or failure by capturing in a document theassociated inputs, state changes, outputs, and result status. Whenoperated over a series of test cases that exercises every possible paththrough an application or application component, this record mayrepresent both a complete specification of the object to be verified27510 and a log of its verification. FIG. 6 and its description depictthe process associated with this module.

Within test plan database 27550, coverage tests 27554 may augment theprior tests with cases specifically targeted at executing every logicpath in an object to be verified 27510. Such tests may be created by theusual method in application factory system 27100, which is to record amanual test in the interactive verification framework 27530. Inaddition, coverage tests 27554 may be generated automatically using aspecific pass through build engine 27140 (whose details are inheritedfrom U.S. Pat. No. 8,719,776 with respect to its FIG. 4, for example,subsystem linking services, test coverage generator compiler, etc.) inwhich target device specific language code generator 435 and SDK 440 areselected for the purpose. In this case the precompiled component objects436 may be existing test case templates, and the SDK 440 may includetools for code coverage tracing and input fuzzing in support ofautomatic test case generation; such tools have become common for mostprogramming languages and may be incorporated as enhancements notdepicted in build engine 27140. The target device specific executable412 in this case is the set of test cases assembled into a sequence thatcovers all possible inputs and decisions; when the studio user requeststhis specific build target, the result may as usual be automaticallyhanded off to verification engine 27150, which in turn may recognize itas a test case set and store it in plan database 27550 under coveragetests 27554.

Also within test plan database 27550, document templates 27555 mayprovide standard formats for documents that are generated automatically.A coordinator user may select a particular template into which thedocument generation procedure (see FIG. 6 ) places its results, therebyallowing for different context, different header information, ordifferent section layout depending on the selected template.

While the examples shown herein may be illustrated as functions,operations, or desired results as individual modules or separateelements, one of ordinary skill in the art would recognize that one ormore of these modules may be combined into a single functional block orelement, One of ordinary skill in the art would also recognize that asingle module may be divided into multiple modules.

III. Methods of Operation

FIG. 5 illustrates a process distributed among a coordinatorapplication, coordinator portal, and other elements of a trialoperations service suite in a clinical trial operations system accordingto an exemplary embodiment of the automatic documentation aspect.Create/edit documentation process 5701 is a variation on coordinatordesign trial process 5700 in the base patent application(s). Usingcreate/edit documentation process 5701, a coordinator may direct thesystem 100 to document itself (create) and add or rearrange text in thatdocument (edit). Here again, the three-actor model andtemporal/informational interaction conventions used throughout thisspecification and the base patent application(s) apply. Thesub-processes in this case are labeled user actions 5705, applicationactions 5710, and service actions 5715 the primary service actor istrial design services 271 and the application factory system 27100embedded in it.

Create/edit documentation process 5701 begins with the prerequisite ofcoordinate logon process 5500, in which all three actors participate.Coordinator logon process 5500 is not described in the presentspecification, being inherited from and identical to the process of thatname in the base patent application(s). After that, at operation 5720the coordinator may select trial design services 271 from the menu ofcoordinator portal services 270. At operation 5735 coordinatorapplication 170 may get the corresponding additional screens and logicfrom coordinator portal 207, which would at operation 5755 provide a webapp implementing the composition studio 27120 of application factorysystem 27100. At operation 5740, coordinator application 170 may presentthe screens and controls constituting the user interface of saidcomposition studio 27120, enabling the coordinator at operation 5721 torequest automatic document creation. This request may be particularizedto indicate either Auto-IRB or Auto-Spec as the document to create, orit may be unparticular causing both document types to be created. Therequest may also specify a particular template for the document ifmultiple templates are available from document templates module 27555.Coordinator application 170 then relays the request to the service atoperation 5741.

At operation 5761, composition studio 27120 may collaborate withverification engine 27150 as described in the context of FIG. 4 togenerate the requested document(s), using the method(s) detailed in FIG.6 and/or FIG. 7 . Once the requested document(s) is/are generated andmade available, the document editor 27533 in interactive verificationframework 27530 may collaborate with coordinator application 170 so thatat operation 5742 the latter may present the document(s) to thecoordinator user with controls for annotating and arranging thematerial. Such controls may take the form of a commonly-availabledocument editing user interface, although the application and servicemay place constraints on what can be changed in order to preserve theintegrity of the automatically generated content and prevent falsifyingthe actual design. For example, the annotation editor may preventplacement of text or imagery on top of a captured screen image in such away that the screen content appears altered, and it may prevent deletionof text, allowing only text additions, section rearrangement withoutseparating images from accompanying text, and font adjustments.

In operation 5722, then, the coordinator user may exercise theannotation editor's controls making permitted changes as needed tocomplete the document according to the goal, for example to satisfy anIRB or other regulatory review. Upon submission of such changes,application 170 may then at operation 5743 relay them to the service,which at operation 5762 may save the changes and produce the finaldocuments for distribution via distribution center 27130. Additionalactions that are not depicted may include copying any changes thatannotate system behaviors or screen elements back into the correspondingbehavior model or screen design source as comments; interpreting anyquestions placed in annotations as possible bug reports or featurerequests and flagging them for developer review via a configurationcontrol subsystem embedded in application factory system 27100, whichmay be an enhanced function of distribution center 27130 via its libraryservices module 310 as shown in U.S. Pat. No. 8,719,776; and recordingversion control information associated with the change submission andrelated actions, which also may utilize a configuration controlsubsystem embedded as noted in application factory system 27100.

Though not depicted explicitly, iterations over this basic sequence mayoccur as needed, Once all iterations have been performed, create/editdocumentation process 5701 ends with each sub-process quiescing in itsown way. User sub-process 5705 simply finishes at operation 5730, withnothing further for the coordinator to do. In application sub-process5710, coordinator application 170 remains running in coordinatorworkstation 107, so its quiescent state is waiting for the user's nextselection at operation 5750. Similarly, coordinator portal 207 has anestablished session for this user, with coordinator application 170connected and authenticated, so it too quiesces in an active statewaiting for the next application request at operation 5765. Any of theother coordinator methods may be selected in this state.

FIG. 6 illustrates a method whereby a trial operations service suite ina clinical trial operations system may automatically examine the trialdesign and generate a trial protocol description suitable for IRBsubmission. The method may be executed as part of the processing duringoperation 5761 of create/edit documentation process 5701, described inthe context of FIG. 5 above. Produce trial protocol description documentmethod 8600 begins with initialization of a document template atoperation 8610. The document template may have been selected by a useras part of operation 5721 of create/edit documentation process 5701,described in the context of FIG. 5 above. Alternatively, the documenttemplate may be the only one available in document templates module27555, or it may be selected automatically by matching a contextvariable related to one or more trial-specific factors such as corporateor divisional branding, type or phase of study, class of drug, device,or procedure being studied, number of participants, country or languageof participants, etc.

IRB documentation describes the trial protocol with respect to itsinteractions with study subjects so that the board may determinecompliance with research ethics, laws, and sound scientific methods. Inthis context the target document is essentially a specification ofpatient application 140 covering its screens, inputs, and screen toscreen flow. These are all elements of the design as created by acoordinator using trial design services 271, expressed in the visualrepresentation language when shown to the coordinator user in thecomposition studio, and expressed in the XML-based representationlanguage internally. Method 8600 may document this design by firstiterating over the internal representation of the patient application'sworkflows applying generate workflow traversal paths subroutine 8620 toeach one, thereby producing for each workflow a set of paths through itbased on the decisions in it and the range of possible inputs that maybe provided by the patient user at each decision point. Inputs mayinclude touch or click locations, gestures, character field valuesentered using a virtual keyboard, numerical values entered using asliding selector, speech entered via a microphone, or any other piece ofinformation that any sensor may collect under control of the user.Stored data, which may when executing come from a sensor not directlycontrolled by the user or be provided by an element of patient portalservices 240, may also be considered by workflow logic in establishing apath.

When at operation 8630 there are no more workflows left to process inthe design for patient application 140, method 8600 may then iterateover the resulting traversal paths applying to each the simulate pathand capture result subroutine 8640, thereby inserting into the Auto-IRBdocument a representation of the path that includes screens displayedand corresponding user inputs.

Note that in general, the number of such paths may be quite large due tothe range of possible inputs at each decision point, yet for any givenworkflow the actual number of distinct paths may be relatively small bydesign. Refer to FIG. 8 and FIG. 9 for an example. Therefore, afterprocessing a path through subroutine 8640, method 8600 may at operation8650 filter the document for duplicate results in terms of screencontent and flow shape. That is, when the resulting document section fora path features the same screens presented in the same order as a pathpreviously accepted into the document, the filter may combine the newsection into the previous one by first adding the input set producingthe new path to the corresponding input description of the existingpath, and then discarding the path and screen images. Detection of aduplicate may apply a variety of techniques with different complexities,from matching identifiers coded with each screen template in the designto recognizing similar images using artificial intelligence algorithms,depending on the intricacy of the actual workflows and screens in thedesign. Where a screen template contains a field for showing variableoutput based on a database entry or previous user input, the filter mayemploy a mask to ignore that field when comparing images. The range ofpossible techniques is not limited to these examples.

Finally, when at operation 8660 there are no more traversal paths tocapture and filter, method 8600 comes to an end at termination point8670.

Subroutine 8620 generate workflow traversal paths is expanded in FIG. 6to show its detailed processing. Refer also to FIG. 8 and FIG. 9 , whichdepict an example workflow and its traversal paths. Beginning atoperation 8621, the subroutine may generate a graph of the mainline flowassociated with the workflow being examined, through to the firstdecision point—that is, a point in the flow that may branch or continuebased on the value of a data item or user input.

At this point the subroutine enters a block of operations that may berepeated for every such decision point in every branch of the workflow.At operation 8622 a subgraph may be constructed for the subflow orsubflows that may follow from the decision point at hand; these are thenadded to the overall flow graph. At operation 22023 any screen that maybe associated with this decision point, if it is a wait state at whichuser input may be expected, may be linked to the flow graph at thispoint. Operation 8624 examines the expected user input(s) and data itemson which this decision point depends, analyzes the logic encoded in theworkflow at this point, constructs the set of possible combinations thatma occur, and then records a distinct path through the flow associatedwith each of the input combinations. As part of recording a path, itsinput and data combinations may be generalized into range combinationsin order to reduce the number of discrete path entries created; everyvalue in each range combination so generalized would follow the samepath and/or display the same screen as a result. To the extent thisoperation can generalize range combinations, the number of pathsrecorded may be reduced, in turn lightening the load on (andcorresponding complexity of) the duplicate results filter in operation8650 of the main method 8600.

When operation 8625 determines that the workflow has no more decisionpoints to analyze, subroutine 8620 reaches its end and returns viaoperation 8626 to the main method.

Subroutine 8640 simulate path and capture result is expanded in FIG. 6to show its detailed processing. Beginning at operation 8641, thesubroutine may draw into the document the flow graph associated with thetraversal path at hand The procedure for drawing may be parameterizedsuch that the size of the resulting diagram is scaled to be readable.For example, if only a few screens and transitions occur in the flow asingle diagram on a single page may be produced, but beyond a thresholdmultiple diagrams linked across multiple pages may be produced. Thethreshold may be as simple as a single parameter controlling the numberof screens that maybe shown on one page; this parameter may be tunableby a coordinator user as part of the trial design or document creationrequest, or set to a default value in the implementation of this method.More complex parameterizations may also be applied, such as onedepending on both the number of screens and the number of branches fromeach screen based on the variety of user inputs required on each screen.

Once the main flow has been drawn, the subroutine enters a block ofoperations that may be repeated for every decision point or user inputwait state in every branch of the path. At operation 8642 a simulationof the path is executed that traverses the path's flow graph to an inputwait point. If any decisions are made along the way based on data thatis not taken from user input, these are documented as well by noting thedata values that cause the flow to follow this particular path. Thescreen associated with this input wait is then drawn into'the documentat the appropriate point along the flow graph drawing, and annotatedwith the input values used at this point in this particular path. Thoseinputs are then injected into the simulation. As long as operation 8643determines that there are more wait points to be examined in the path,operation 8642 is repeated for each one. When all branches of the pathhave been traversed and captured in the document, operation 8643 dropsout of the loop and the subroutine ends at operation 8644 by returningto the main method.

FIG. 7 illustrates a method whereby atrial operations service suite in aclinical trial operations system may automatically examine the trialdesign and generate a software specification for the service suiteitself and the applications that interact with it. The method may beexecuted as part of the processing during operation 5761 of create/editdocumentation process 5701, described in the context of FIG. 5 above.Produce system specification document method 8700 begins withinitialization of a document template at operation 8710. The documenttemplate may have been selected by a user as part of operation 5721 ofcreate/edit documentation process 5701, described in the context of FIG.5 above. Alternatively, the document template may be the only oneavailable in document templates module 27555, or it may be selectedautomatically by matching a context variable related to one or morespecific factors such as corporate or divisional branding, whethercertain gateways and services are enabled or disabled, country orlanguage of the coordinator, etc.

A system specification describes the entire clinical trial operationssystem 100, including all aspects of the trial operations service andeach application In this context the target document is a superset ofthe Auto-IRB document, covering not just patient application 140 butalso clinician application 150, investigator application 160,coordinator application 170, and trial operations service suite 200.Further, where the Auto-IRB document covers only the patient-facingaspects of patient application 140, the Auto-Spec document also coversservice-facing aspects such as interactions between application andportal—both behaviors and data structures. In general, anything in thesystem may be changed by a coordinator using trial design services 271,but in practice very little is likely to be customized except patientapplication 140. Therefore, the Auto-Spec template may provide a muchlarger portion of the actual specification in this case. For services,gateways, and analytics in particular, the primary customizations maysimply be to enable or disable particular features. Templateinitialization at operation 8710 may take this into consideration byremoving entire sections for disabled features.

The remainder of produce system specification document method 8700 maythen be very similar to produce trial protocol description documentmethod 8600, with an additional loop to cover the other system elements.That is, method 8600 may document the customizations made in eachelement of the system by first iterating over the internalrepresentation of workflows and logic for each application, portalservice, relay service, gateway, and analytics module, applying generateworkflow traversal paths subroutine 8620 to each one, thereby producingfor each workflow a set of paths through it based on the decisions in itand the range of possible inputs that may be provided by a user at eachdecision point. Inputs may include touch or click locations, gestures,character field values entered using a virtual keyboard, numericalvalues entered using a sliding selector, speech entered via amicrophone, or any other piece of information that any sensor maycollect under control of the user. Stored data, which in an executingapplication may come from a sensor not directly controlled by the useror be provided by an element of its corresponding portal, and which inan executing element of trial operations service, suite 200 may comefrom operational databases 202 or security configuration and managementauthentication database 201, may also be considered by workflow logic inestablishing a path.

In addition to generating the traversal paths for each workflow in thesystem, any data schemata associated with the workflow at hand arerecorded at operation 8720. This ensures that database tables and fieldsused in the workflow, and its logic are documented, as well as datastructures for inter-process and inter-element communication, usernotifications, and inter-system communication via gateways.

When at operation 8730 there are no more workflows left to process inthe design for the element at hand, and at operation 8735 there are nomore applications, services, gateways, or analytics modules left toprocess in the design for the system, method 8700 may then iterate overthe resulting traversal paths applying to each the simulate path andcapture result subroutine 8640, thereby inserting into the Auto-Specdocument a representation of the path that includes screens displayedand corresponding user inputs, plus additional information capturedalong the way including the aforementioned data schemata.

Note that in general, the number of such paths may be quite large due tothe range of possible inputs at each decision point, yet for any givenworkflow the actual number of distinct paths may be relatively small bydesign. Therefore, after processing a path through subroutine 8640,method 8700 may at operation 8750 filter the document for duplicateresults in terms of screen content and flow shape, as well as, in termsof data schemata content or shape where appropriate, such as in anelement with inter-process, inter-element, or inter-system communicationbehaviors. Similar comparison techniques may be applied here aspreviously described for operation 8650 in method 8600.

Finally, when at operation 8760 there are no more traversal paths tocapture and filter, method 8700 comes to an end at termination point8770.

One skilled in the art may observe that the resulting document is not aspecification in the traditional sense—a relatively compact set ofrequirements for what the system should do. Instead, because theembedded application factory system 27100 enables the system's designand implementation to be conflated, it is actually a description, usingthe specification form, of what the system does. Thus, the Auto-Specdocument may be used for oversight to understand the system, but it willnot be needed for driving a development project separate from the designproject. That latter endeavor having been eliminated entirely by thistechnology is the primary benefit of the embodiments in both the baseand present patent applications.

FIG. 8 and FIG. 9 illustrate an exemplary workflow and its traversalpaths as may be generated by subroutine 8620 generate workflow traversalpaths. Example work flow 8801 depicts what may be a structure for aparticipant qualification workflow, wherein a patient candidate maydetermine whether he or she is a fit for a specific clinical trialsupported by a particular instance of system 100. Note that theassociated user interface screens for example workflow 8801 are notdepicted, but would certainly exist and be processed by methods 8600 and8700, having been created using the visual language described in thecontext of example user interface 390 in FIG. 3 . One skilled in the artwill recognize the visual language of example workflow 8801 from itsearlier explication using example workflow 380 in FIG. 3 , so thedetailed flow is not described further other than to point out thatexample workflow 8801 has a richer structure with more decision pointsthan example workflow 380, and is thereby more suitable for depicting avariety of traversal paths.

Example workflow 8801 is duplicated in miniature as example workflowcopy 8802, onto which traversal path 8821 is overlaid as a heavy arrowflowing from its start symbol to its end symbol through the firstdecision point in the flow. FIG. 9 then depicts six more miniaturecopies of example workflow 8801, shown as example workflow copies 8803,8804, 8805, 8806, 8807, and 8808. These are in turn overlain by theirrespective traversal paths 8831, 8841, 8851, 8861, 8871, and 8881. Thussubroutine 8620 generate workflow traversal paths would record sevendistinct traversal paths during its execution over example workflow8801, covering every possible branch through the workflow based ondifferent user inputs or calculated outcomes at each decision point.

The processes of FIG. 5 through FIG. 9 may be implemented, in variousdifferent ways without departing from the scope of the disclosure. Forinstance, the operations ma be performed in different orders than shown.As another example, some processes may include additional operationsand/or omit various listed operations.

While the examples shown may illustrate many individual modules (oroperation sets, performed by software instructions and/or user,clinician, operator, etc. action) as separate elements, one of ordinaryskill in the art would recognize that these modules may be combined intoa single functional block or element. One of ordinary skill in the anwould also recognize that a single module may be divided into multiplemodules. Further, the steps, operation sets, functions, etc. describedin the various modules are understood to be actionable via a computer orprocessor executing instructions, to provide the features and resultsdescribed herein. Thus, one of ordinary skill in the computer andsoftware arts, having read this disclosure, could devise and replicatethe features of the various modules, as well as modifications within thescope of this disclosure without undue experimentation.

IV. Computer System

Many of the processes and modules described above may be implemented assoftware processes that are specified as one or more sets ofinstructions recorded on a non-transitory storage medium. When theseinstructions are executed b one or more computational element(s)microprocessors, microcontrollers, digital signal processors (DSPs),application-sped fic integrated circuits (ASICs), field programmablegate arrays (FPGAs), etc.) the instructions cause the computationalelement(s) to perform actions specified in the instructions.

In some embodiments, various processes and modules described above maybe implemented completely using electronic circuitry that may includevarious sets of devices or elements (e.g., sensors, logic gates, analogto digital converters, digital to analog converters, comparators, etc.).Such circuitry may be able to perform functions and/or features that maybe associated with various software elements described throughout.

FIG. 78 in the base patent application(s) illustrates a schematic blockdiagram of an exemplary computer system 7800 used to implement someembodiments. For example, the systems described above in reference toFIG. 1 through FIG. 4 may be at least partially implemented usingcomputer system 7800. As another example, the processes described inreference to FIG. 5 through FIG. 9 may be at least partially implementedusing sets of instructions that are executed using computer system 7800.

Computer system 7800 may be implemented using various appropriatedevices For instance, the computer system may be implemented using oneor more personal computers (PCs), servers, mobile devices (e.g., asmartphone), tablet devices, and/or any other appropriate devices. Thevarious devices may work alone (e.g., the computer system may beimplemented as a single PC) or in conjunction (e.g., some components ofthe computer system may be provided by a mobile device while othercomponents are provided by a tablet device).

FIG. 78 of the base patent application(s) and computer system 7800 areconsidered to be incorporated in the present patent application byreference, and therefore are not detailed here.

As used in this specification and any claims of this application, theterms “computer”, “server”, “processor”, and “memory” all refer toelectronic devices. These terms exclude people or groups of people. Asused in this specification and any claims of this application, the term“non-transitory storage medium” is entirely restricted to tangible,physical objects that stole information in a form that is readable byelectronic devices. These terms exclude any wireless or other ephemeralsignals.

It should be recognized by one of ordinary skill in the art that any orall of the components of computer system 7800 may be used in conjunctionwith some embodiments. Moreover, one of ordinary skill in the art willappreciate that many other system configurations may also be used inconjunction with some embodiments or components of some embodiments.

In addition, while the examples shown may illustrate many individualmodules as separate elements, one of ordinary skill in the art wouldrecognize that these modules may be combined into a single functionalblock or, element. One of ordinary skill in the art would also recognizethat a single module may be divided into multiple modules.

The foregoing relates to illustrative details of exemplary embodimentsand modifications may be made without departing from the scope of thedisclosure as defined by the following claims.

1-22. (canceled)
 23. A method comprising: providing acoordinator-customized patient application configured to access apatient portal for a clinical trial; iterating over a representation ofworkflows of the coordinator-customized patient application, eachworkflow based on decision points and a range of possible inputs at eachdecision point of the decision points; iterating, in response toiterating traversal paths over the representation of workflows, asimulated path over resulting traversal paths applying to each workflow;and inserting into a review board document, a representation of anytraversal path that includes user screens for the clinical trial andcorresponding user inputs as determined by the simulated path.
 24. Themethod of claim 23, further comprising selectively generating aspecification document describing a design of the clinical trial. 25.The method of claim 23, further comprising generating, using a workflowpalette and a visual programming toolkit, the user screens for theclinical trial.
 26. The method of claim 25, further comprising:capturing in a specification document, associated inputs, state changes,outputs, and result status for test cases; executing at least some logicin an object to be verified; accessing an interactive verificationframework with a document editor and supporting user interface to editthe specification document; using a test device library containingtestable device models to test at least one device model; and verifyingapplication objects and application components.
 27. The method of claim26, further comprising providing compilation and subsystem linkingservices using a build engine.
 28. The method of claim 26, furthercomprising accessing a composition studio configured to produceapplications and the application components.
 29. The method of claim 28,further comprising providing a document creation request control.
 30. Asystem comprising: one or more computational elements; and anon-transitory storage medium including instructions executable by theone or more computational elements to configure the one or morecomputational elements to: provide a coordinator-customized patientapplication configured to access a patient portal for a clinical trial;iterate over a representation of workflows of the coordinator-customizedpatient application, each workflow based on decision points and a rangeof possible inputs at each decision point of the decision points;iterate, in response to iterating traversal paths over therepresentation of workflows, a simulated path over resulting traversalpaths applying to each workflow; and insert into a review boarddocument, a representation of any traversal path that includes userscreens for the clinical trial and corresponding user inputs asdetermined by the simulated path.
 31. The system of claim 30, whereinthe instructions are executable to configure the one or morecomputational elements to selectively generate a specification documentdescribing a design of the clinical trial.
 32. The system of claim 30,wherein the instructions are executable to configure the one or morecomputational elements to generate, using a workflow palette and avisual programming toolkit, the user screens for the clinical trial. 33.The system of claim 32, wherein the instructions are executable toconfigure the one or more computational elements to: capture in aspecification document, associated inputs, state changes, outputs, andresult status for test cases; execute at least some logic in an objectto be verified; access an interactive verification framework with adocument editor and supporting user interface to edit the specificationdocument; use a test device library containing testable device models totest at least one device model; and verify application objects andapplication components.
 34. The system of claim 33, wherein theinstructions are executable to configure the one or more computationalelements to provide compilation and subsystem linking services using abuild engine.
 35. The system of claim 33, wherein the instructions areexecutable to configure the one or more computational elements to accessa composition studio configured to produce applications and theapplication components.
 36. The system of claim 35, wherein theinstructions are executable to configure the one or more computationalelements to provide a document creation request control.
 37. Anon-transitory storage medium including instructions executable by oneor more computational elements to perform operations comprising:providing a coordinator-customized patient application configured toaccess a patient portal for a clinical trial; iterating over arepresentation of workflows of the coordinator-customized patientapplication, each workflow based on decision points and a range ofpossible inputs at each decision point of the decision points;iterating, in response to iterating traversal paths over therepresentation of workflows, a simulated path over resulting traversalpaths applying to each workflow; and inserting into a review boarddocument, a representation of any traversal path that includes userscreens for the clinical trial and corresponding user inputs asdetermined by the simulated path.
 38. The non-transitory storage mediumof claim 37, wherein the instructions are further executable to performoperations comprising generating a specification document describing adesign of the clinical trial.
 39. The non-transitory storage medium ofclaim 37, wherein the instructions are further executable to performoperations comprising using a workflow palette and a visual programmingtoolkit, the user screens for the clinical trial.
 40. The non-transitorystorage medium of claim 39, wherein the instructions are furtherexecutable to perform operations comprising: capturing in aspecification document, associated inputs, state changes, outputs, andresult status for test cases; executing at least some logic in an objectto be verified; accessing an interactive verification framework with adocument editor and supporting user interface to edit the specificationdocument; using a test device library containing testable device modelsto test at least one device model; and verifying application objects andapplication components.
 41. The non-transitory storage medium of claim40, wherein the instructions are further executable to performoperations comprising providing compilation and subsystem linkingservices using a build engine.
 42. The non-transitory storage medium ofclaim 40, wherein the instructions are further executable to performoperations comprising accessing a composition studio configured toproduce applications and the application components.